"""
Comprehensive EXIF Data Extractor - Maximum Metadata Extraction
================================================================
File      : 01-extract-exif-summary-V05.py
Version   : V05
Author    : Yahya Nazer
Copyright : (c) 2025 Chatbizdb.com - Yahya Nazer
Date      : 2026-05-13

Objective
---------
Let the user select a folder containing images, recursively scan it,
extract every available EXIF and metadata tag using up to four libraries
in priority order, and write four output files to a timestamped report
folder beside the selected image folder.

Folder Layout
--------------
    Selected image folder  :  TOP/Data/
    TOP folder (parent)    :  TOP/
    Report root            :  TOP/C-Reports/
    Timestamped output     :  TOP/C-Reports/exif-report-YYYY-MM-DD--HH-MM/

    Output files:
        comprehensive_summary.csv      -- One column per discovered EXIF key.
        organized_summary.csv          -- Columns ordered by mapping category.
        metadata_index.html            -- Clickable HTML table with Camera/Date.
        metadata_fields_reference.txt  -- Plain list of all discovered keys.

    Mapping file (optional, must sit BESIDE this .py script):
        Metadata-Field-Mapping.csv     -- CSV with columns:
            Metadata Field | Windows Properties Name | Category

Four-Library Extraction Strategy
----------------------------------
Metadata is extracted by running up to four libraries in sequence.  Each
library may find tags the others miss.  The FIRST library to write a key
wins (first-write-wins policy enforced by add_metadata_value):

    Priority    Library     Description
    --------    -------     -----------------------------------------------
    1 (always)  Pillow      PIL / Image.getexif() -- standard JPEG/TIFF EXIF.
    2 (always)  exifread    Deep tag reader; handles maker notes, GPS, more.
    3 (optional)piexif      Structured IFD access (0th/Exif/GPS/1st/Interop).
                            Only for JPEG/TIFF; skipped if not installed.
    4 (optional)rawpy       RAW camera file metadata (CR2/NEF/ARW/DNG etc.).
                            Skipped if not installed or file is not RAW.

    Install all four:
        pip install Pillow exifread piexif rawpy

EXIF Data Concepts
-------------------
EXIF (Exchangeable Image File Format) is a standard for embedding metadata
in image files.  Key concepts used in this script:

    IFD (Image File Directory)
        A structured block inside a JPEG/TIFF containing tag-value pairs.
        piexif exposes five IFDs: '0th' (basic), 'Exif' (exposure details),
        'GPS' (location), '1st' (thumbnail), 'Interop' (interoperability).

    Tag ID vs Tag Name
        Raw EXIF stores numeric IDs (e.g. 0x010F for Make, 0x0110 for Model).
        ExifTags.TAGS maps numeric ID -> human-readable string name.
        ExifTags.TAGS.get(tag_id, tag_id) returns the name if known, or the
        raw integer if the tag is not in the lookup table.

    IFDRational / Fraction values
        Exposure, focal length, GPS coordinates etc. are stored as fractions
        (numerator, denominator) in EXIF to preserve precision.
        fraction_to_float() converts (num, den) tuples to Python floats.

Fuzzy Key Matching in get_mapped_value()
-----------------------------------------
Different libraries store the same data under different key names.  For
example, the shutter speed may appear as:
    'ExposureTime'    (Pillow / piexif)
    'EXIF ExposureTime' (exifread)
    'exposure_time'   (rawpy)

_normalize_key_for_match() strips spaces, dots, colons, and underscores,
then lowercases the result, so all three variants normalise to
'exposuretime' and the mapping file can resolve them with one entry.

Lookup order in get_mapped_value():
    1. Exact match on metadata_field (field name from mapping CSV).
    2. Exact match on windows_name   (display name from mapping CSV).
    3. Fuzzy match on metadata_field (after normalisation).
    4. Fuzzy match on windows_name   (after normalisation).

First-Write-Wins Policy
------------------------
add_metadata_value() only writes a key if it is NOT already in the dict:
    if key not in target:
        target[key] = value
This means Pillow's value for 'Make' is kept even if exifread and piexif
also report 'Make'; no duplicate or override occurs.

Supported Image Extensions
----------------------------
Standard formats: .jpg .jpeg .png .gif .bmp .tiff .tif .webp
RAW camera formats: .cr2 .cr3 .nef .arw .dng .orf .rw2 .pef .srw

Dependencies
-------------
Required:
    pip install Pillow exifread

Optional (silently skipped if missing):
    pip install piexif rawpy

Standard library only (no extra install):
    os, sys, csv, traceback, pathlib.Path, datetime

Function Summary
-----------------
    Function                            Purpose
    --------------------------------    -----------------------------------------
    setup_utf8()                        Reconfigure stdout/stderr to UTF-8.
    select_folder()                     GUI folder-picker with CLI fallback.
    create_timestamped_folder()         Create timestamped output subfolder.
    get_image_files()                   Recursively collect image file paths.
    fraction_to_float()                 Convert (num, den) EXIF tuples to float.
    normalize_tag_name()                Decode bytes tag names to strings.
    add_metadata_value()                Safely add a key to a metadata dict
                                        (first-write-wins, decode bytes).
    extract_with_pil()                  Extract EXIF using Pillow.
    extract_with_piexif()               Extract EXIF using piexif (optional).
    extract_with_exifread()             Extract EXIF using exifread.
    extract_with_rawpy()                Extract RAW metadata using rawpy (opt).
    extract_all_metadata()              Run all extractors; merge into one dict.
    load_field_mapping()                Load Metadata-Field-Mapping.csv.
    format_value()                      Format a metadata value for CSV/HTML.
    _normalize_key_for_match()          Strip punctuation/case for fuzzy match.
    get_mapped_value()                  Resolve a mapping row to a value.
    create_comprehensive_summary_csv()  Write comprehensive_summary.csv.
    create_organized_summary_by_category() Write organized_summary.csv.
    create_html_metadata_index()        Write metadata_index.html.
    open_html_report()                  Open HTML in default browser/viewer.
    main()                              Orchestrate all steps end-to-end.

Step-by-Step Flow (inside main())
------------------------------------
Step 1  : User selects image folder via GUI dialog or console input.
Step 2  : Derive TOP folder (parent of selected), create Report/ root.
Step 3  : Create timestamped EXIF-Summary-* subfolder inside Report/.
Step 4  : Load Metadata-Field-Mapping.csv from the program folder (optional).
Step 5  : Discover all image/RAW files recursively inside the image folder.
Step 6  : Extract metadata from every image using all available libraries.
Step 7  : Write comprehensive_summary.csv and metadata_fields_reference.txt.
Step 8  : Write organized_summary.csv (only if mapping is loaded).
Step 9  : Write metadata_index.html (only if mapping is loaded).
Step 10 : Open metadata_index.html in the default browser.
"""

# ===========================================================================
# Imports
# ===========================================================================
import os                           # File system operations and path handling
import sys                          # stdout/stderr reconfiguration
import csv                          # CSV reader/writer
import traceback                    # Detailed exception stack trace printing
from pathlib import Path            # Modern object-oriented path handling
from datetime import datetime       # Timestamp generation

# Required imaging libraries
from PIL import Image, ExifTags     # Pillow: EXIF extraction and tag name lookup

import exifread                     # Deep EXIF reader: maker notes, GPS, raw tags

# ===========================================================================
# Optional library imports with silent fallback to None
#
# Setting the module name to None when the import fails allows all
# downstream code to check 'if piexif is None' / 'if rawpy is None'
# and skip silently rather than raising NameError at call time.
# ===========================================================================

# piexif: structured IFD-level access (JPEG/TIFF only)
try:
    import piexif
except ImportError:
    piexif = None

# rawpy: RAW camera file reader (CR2, NEF, ARW, DNG, etc.)
try:
    import rawpy
except ImportError:
    rawpy = None


# ===========================================================================
# Function: setup_utf8()
# ===========================================================================
def setup_utf8() -> None:
    """
    Reconfigure stdout and stderr to UTF-8 encoding.

    Camera-generated filenames and EXIF strings can contain accented or
    non-ASCII characters.  Without UTF-8, printing such strings on a
    Windows terminal raises UnicodeEncodeError.

    The hasattr() guard ensures this works on Python 3.6 where reconfigure()
    does not yet exist.  Failures are silently ignored -- the script
    continues with whatever encoding is already active.
    """
    try:
        if hasattr(sys.stdout, 'reconfigure'):
            sys.stdout.reconfigure(encoding='utf-8')
        if hasattr(sys.stderr, 'reconfigure'):
            sys.stderr.reconfigure(encoding='utf-8')
    except Exception:
        pass    # Non-critical; continue without reconfiguration


# Run immediately so all subsequent prints use UTF-8
setup_utf8()


# ===========================================================================
# Function: select_folder()
# ===========================================================================
def select_folder() -> Path:
    """
    Ask the user to select the folder containing images.

    Attempts to open a tkinter GUI folder-picker dialog.  If tkinter is
    unavailable (headless server, minimal Linux) or the dialog fails, falls
    back to a console prompt asking for a typed path.

    sys.exit(1) is called for all unrecoverable conditions (no selection,
    invalid path) so the caller never receives an invalid Path object.

    Returns:
        Path: Validated absolute path of the selected image folder.
    """
    print('[INFO]  Please select the folder containing your images.')
    print('[INFO]  Tip: select the DATA folder, e.g. TOP/Data')

    try:
        import tkinter as tk
        from tkinter import filedialog

        root = tk.Tk()
        root.withdraw()
        folder = filedialog.askdirectory(title='Select folder with images')
        root.destroy()

        if not folder:
            print('[ERROR] No folder selected. Exiting.')
            sys.exit(1)

        path = Path(folder)
        if not path.is_dir():
            print('[ERROR] Selected path is not a folder. Exiting.')
            sys.exit(1)

        print(f'[INFO]  Selected image folder: {path}')
        return path

    except Exception as e:
        # tkinter unavailable or dialog failed -- CLI fallback
        print(f'[WARN]  GUI dialog not available ({e}). Falling back to console input.')
        folder = input('  Enter full path to folder with images: ').strip()

        if not folder:
            print('[ERROR] No path provided. Exiting.')
            sys.exit(1)

        path = Path(folder)
        if not path.is_dir():
            print(f'[ERROR] Path is not a valid folder: {path}. Exiting.')
            sys.exit(1)

        print(f'[INFO]  Selected image folder: {path}')
        return path


# ===========================================================================
# Function: create_timestamped_folder()
# ===========================================================================
def create_timestamped_folder(base_folder: Path,
                               prefix: str = 'EXIF-Summary') -> Path:
    """
    Create and return a timestamped subfolder inside base_folder.

    Folder name format:  <prefix>-YYYY-MM-DD--HH-MM
    Example:             exif-report-2026-05-13--14-30

    A new folder is created for every run so previous reports are never
    overwritten.  parents=True also creates base_folder if it does not yet
    exist.

    Args:
        base_folder (Path): Parent folder in which the subfolder is created.
        prefix      (str):  Prefix string for the folder name.

    Returns:
        Path: Absolute path of the newly created timestamped folder.
    """
    timestamp = datetime.now().strftime('%Y-%m-%d--%H-%M')
    out = base_folder / f'{prefix}-{timestamp}'
    out.mkdir(parents=True, exist_ok=True)
    print(f'[INFO]  Output folder: {out}')
    return out


# ===========================================================================
# Supported image file extensions
#
# Standard formats use Pillow + exifread.
# RAW formats additionally attempt rawpy extraction.
# ===========================================================================
IMAGE_EXTENSIONS = {
    # Standard compressed/lossless formats
    '.jpg', '.jpeg', '.png', '.gif', '.bmp',
    '.tiff', '.tif', '.webp',
    # RAW camera formats
    '.cr2', '.cr3',   # Canon
    '.nef',           # Nikon
    '.arw',           # Sony
    '.dng',           # Adobe Digital Negative (universal RAW)
    '.orf',           # Olympus
    '.rw2',           # Panasonic
    '.pef',           # Pentax
    '.srw',           # Samsung
}

# Set of RAW extensions that trigger rawpy extraction
RAW_EXTENSIONS = {
    '.cr2', '.cr3', '.nef', '.arw', '.dng',
    '.orf', '.rw2', '.pef', '.srw',
}


# ===========================================================================
# Function: get_image_files()
# ===========================================================================
def get_image_files(folder_path: Path) -> list:
    """
    Recursively walk folder_path and return a list of supported image paths.

    Path(name).suffix.lower()
        Extracts the file extension from the name string and lowercases it
        so '.JPG' and '.jpg' both match the IMAGE_EXTENSIONS set.

    Files with unsupported extensions are silently skipped.  The function
    prints every found image so the user can verify the scan is working.

    Args:
        folder_path (Path): Root folder to scan.

    Returns:
        list[str]: Absolute path strings for all found image/RAW files.
    """
    image_files = []
    print(f'[INFO]  Scanning for images in: {folder_path}')

    for root, _, files in os.walk(folder_path):
        for name in files:
            if Path(name).suffix.lower() in IMAGE_EXTENSIONS:
                full = str(Path(root) / name)
                image_files.append(full)
                print(f'[DEBUG] Found image: {full}')

    print(f'[INFO]  Total images found: {len(image_files)}')
    return image_files


# ===========================================================================
# Metadata helpers
# ===========================================================================

def fraction_to_float(value):
    """
    Convert a rational (fraction) EXIF value to a Python float when possible.

    EXIF stores many numeric values as fractions to preserve precision.
    For example:
        Exposure time 1/250 s  -> stored as tuple (1, 250)
        Focal length  50 mm    -> stored as tuple (50, 1)
        GPS latitude            -> stored as nested tuples

    Pillow returns these as IFDRational objects or (numerator, denominator)
    tuples depending on the EXIF version.

    This function handles:
        (num, den) tuple  -> float(num) / float(den)  (guard: den != 0)
        Any other type    -> float(value) if possible, else return unchanged.

    The outer try/except returns the original value unchanged for types
    that cannot be meaningfully converted (strings, nested structures).

    Args:
        value: Any EXIF value (tuple, IFDRational, int, float, str, ...).

    Returns:
        float if convertible; otherwise the original value unchanged.
    """
    try:
        if isinstance(value, tuple) and len(value) == 2:
            num, den = value
            if den:                             # Guard: do not divide by zero
                return float(num) / float(den)
        return float(value)
    except Exception:
        return value    # Return unchanged if conversion is not possible


def normalize_tag_name(tag) -> str:
    """
    Normalise an EXIF tag identifier to a readable string key.

    Some libraries return tag names as bytes objects rather than str.
    This function decodes bytes using UTF-8 (ignoring undecodable bytes)
    and converts any other type to str.

    Args:
        tag: Tag name/ID from any EXIF library (str, bytes, int, ...).

    Returns:
        str: Human-readable string representation of the tag name.
    """
    if isinstance(tag, bytes):
        try:
            return tag.decode('utf-8', errors='ignore')
        except Exception:
            return str(tag)
    return str(tag)


def add_metadata_value(target: dict, key, value) -> None:
    """
    Safely insert a metadata key-value pair into the target dictionary.

    Implements the FIRST-WRITE-WINS policy:
        If the key already exists in target, the new value is discarded.
        This ensures Pillow's value for 'Make' is not overwritten by
        exifread's or piexif's value for the same logical tag.

    Processing pipeline applied before insertion:
        1. Return immediately if value is None (nothing to store).
        2. Decode bytes values to UTF-8 strings (repr() fallback).
        3. Convert fraction-like tuples to float via fraction_to_float().
        4. Normalise the key to a string via normalize_tag_name().
        5. Only insert if key is NOT already in target.

    Args:
        target (dict): The metadata dictionary being built.
        key:           Tag name/ID (str, bytes, or int).
        value:         Tag value (any type from EXIF library).
    """
    if value is None:
        return

    # Step 2: decode raw bytes to a readable string
    if isinstance(value, bytes):
        try:
            value = value.decode('utf-8', errors='ignore')
        except Exception:
            value = repr(value)     # Fallback: Python literal representation

    # Step 3: convert EXIF rational fractions to float
    value = fraction_to_float(value)

    # Step 4: normalise key to string
    key = normalize_tag_name(key)

    # Step 5: first-write-wins -- do not overwrite existing entries
    if key not in target:
        target[key] = value


# ===========================================================================
# Per-library extraction functions
# ===========================================================================

def extract_with_pil(image_path: str) -> dict:
    """
    Extract EXIF metadata using Pillow (PIL).

    img.getexif()
        Returns an Exif object: a dict-like mapping of numeric tag IDs to
        their raw values.  Available in Pillow >= 6.0.

    ExifTags.TAGS.get(tag_id, tag_id)
        Maps a numeric tag ID (e.g. 271) to its standard name (e.g. 'Make').
        If the ID is not in the lookup table (proprietary or unknown tags),
        the raw integer is used as the key.

    Pillow handles JPEG, PNG (iTXt/tEXt/zTXt), TIFF, WebP, and BMP.
    It does NOT read RAW files.

    Silent exception handling: if a file cannot be opened (corrupted,
    unsupported variant), an empty dict is returned.

    Args:
        image_path (str): Absolute path to the image file.

    Returns:
        dict: EXIF key-value pairs extracted by Pillow.
    """
    meta = {}
    try:
        with Image.open(image_path) as img:
            exif = img.getexif()
            if exif:
                for tag_id, value in exif.items():
                    # Convert numeric tag ID to its human-readable name
                    tag = ExifTags.TAGS.get(tag_id, tag_id)
                    add_metadata_value(meta, tag, value)
    except Exception:
        pass    # Unsupported format or corrupt file -- return empty dict
    return meta


def extract_with_piexif(image_path: str) -> dict:
    """
    Extract EXIF metadata using piexif (optional, JPEG/TIFF only).

    piexif.load(path)
        Returns a dict with IFD names as keys and tag-value dicts as values.
        The five IFDs accessed:
            '0th'     -- Basic image info (Make, Model, Orientation, ...).
            'Exif'    -- Exposure details (ExposureTime, FocalLength, ...).
            'GPS'     -- Location coordinates (GPSLatitude, GPSLongitude, ...).
            '1st'     -- Thumbnail image data.
            'Interop' -- Interoperability information.

    piexif.TAGS[ifd_name][tag_id]
        Returns a dict with at least a 'name' key: the human-readable tag name.
        Falls back to {"name": str(tag_id)} for unknown tags.

    If piexif is not installed (piexif is None), this function immediately
    returns an empty dict without attempting any operations.

    Args:
        image_path (str): Absolute path to the image file.

    Returns:
        dict: EXIF key-value pairs extracted by piexif.
    """
    meta = {}
    if piexif is None:
        return meta     # Library not installed -- skip silently

    try:
        exif_dict = piexif.load(image_path)

        for ifd_name in ('0th', 'Exif', 'GPS', '1st', 'Interop'):
            if ifd_name in exif_dict:
                for tag_id, value in exif_dict[ifd_name].items():
                    # Look up the human-readable name for this IFD/tag combination
                    tag_info = piexif.TAGS[ifd_name].get(tag_id, {'name': str(tag_id)})
                    tag_name = tag_info['name']
                    add_metadata_value(meta, tag_name, value)

    except Exception:
        pass
    return meta


def extract_with_exifread(image_path: str) -> dict:
    """
    Extract EXIF metadata using the exifread library.

    exifread.process_file(f, details=False)
        Reads the binary file stream and returns a dict of tag name -> IfdTag.
        details=False skips maker note sub-tags for faster processing and
        avoids errors caused by proprietary maker note structures.

        Keys use a 'Section TagName' format, e.g. 'EXIF ExposureTime',
        'Image Make', 'GPS GPSLatitude'.

    str(value)
        IfdTag objects have a __str__ method that returns a human-readable
        representation of the value (e.g. '1/250' for a rational shutter speed).

    exifread is especially strong at:
        - Maker notes (Canon, Nikon, Sony proprietary tags)
        - GPS coordinates
        - Multiple IFDs in a single pass

    Args:
        image_path (str): Absolute path to the image file.

    Returns:
        dict: EXIF key-value pairs extracted by exifread.
    """
    meta = {}
    try:
        with open(image_path, 'rb') as f:       # 'rb' = binary read required for EXIF
            tags = exifread.process_file(f, details=False)
            for tag, value in tags.items():
                add_metadata_value(meta, tag, str(value))
    except Exception:
        pass
    return meta


def extract_with_rawpy(image_path: str) -> dict:
    """
    Extract RAW camera metadata using rawpy (optional, RAW formats only).

    rawpy.imread(path)
        Opens a RAW camera file and returns a RawPy object.
        .metadata contains a RawPyMetaData object with attributes like
        camera_make, camera_model, raw_type, color_count, etc.

    raw.metadata.__dict__
        Python's built-in dict of an object's instance attributes.
        Iterating it gives all metadata attribute names and values.

    Attributes starting with '_' are internal Python private attributes
    (e.g. __class__, __module__) and are skipped with the startswith('_')
    check.

    Keys are prefixed with 'rawpy_' to distinguish them from tags
    extracted by other libraries (avoids key collisions).

    If rawpy is not installed (rawpy is None), returns an empty dict.

    Args:
        image_path (str): Absolute path to the RAW image file.

    Returns:
        dict: RAW metadata key-value pairs prefixed with 'rawpy_'.
    """
    meta = {}
    if rawpy is None:
        return meta     # Library not installed -- skip silently

    try:
        with rawpy.imread(image_path) as raw:
            for k, v in raw.metadata.__dict__.items():
                if not k.startswith('_'):       # Skip Python internal attributes
                    add_metadata_value(meta, f'rawpy_{k}', v)
    except Exception:
        pass
    return meta


# ===========================================================================
# Function: extract_all_metadata()
# ===========================================================================
def extract_all_metadata(image_path: str) -> dict:
    """
    Run all available extractors and merge results into one metadata dict.

    Extraction order and first-write-wins policy:
        1. Pillow     -- always runs first (highest priority).
        2. piexif     -- adds any keys not already found by Pillow.
        3. exifread   -- adds any remaining keys.
        4. rawpy      -- only for RAW extensions; adds rawpy_* prefixed keys.

    Basic file metadata always added last (never overwritten by libraries):
        SourceFile      : Full absolute path to the image file.
        File_Name       : Filename only (no directory).
        File_Size       : File size in bytes.
        File_Extension  : Lowercase extension (e.g. '.jpg').

    Args:
        image_path (str): Absolute path to the image file.

    Returns:
        dict: All discovered metadata tags merged into one flat dictionary.
    """
    all_meta: dict = {}
    print(f'\n[INFO]  Processing: {image_path}')

    # Step 1: Pillow (highest priority -- runs for all formats)
    pil_meta = extract_with_pil(image_path)
    if pil_meta:
        print(f'[DEBUG] Pillow    : {len(pil_meta)} tags found')
    for k, v in pil_meta.items():
        add_metadata_value(all_meta, k, v)

    # Step 2: piexif (optional -- supplements Pillow for JPEG/TIFF)
    pie_meta = extract_with_piexif(image_path)
    if pie_meta:
        print(f'[DEBUG] piexif    : {len(pie_meta)} tags found')
    for k, v in pie_meta.items():
        if k not in all_meta:       # Explicit check mirrors add_metadata_value
            add_metadata_value(all_meta, k, v)

    # Step 3: exifread (supplements both Pillow and piexif)
    exifread_meta = extract_with_exifread(image_path)
    if exifread_meta:
        print(f'[DEBUG] exifread  : {len(exifread_meta)} tags found')
    for k, v in exifread_meta.items():
        if k not in all_meta:
            add_metadata_value(all_meta, k, v)

    # Step 4: rawpy (only attempted for known RAW extensions)
    ext = Path(image_path).suffix.lower()
    if ext in RAW_EXTENSIONS:
        raw_meta = extract_with_rawpy(image_path)
        if raw_meta:
            print(f'[DEBUG] rawpy     : {len(raw_meta)} tags found')
        for k, v in raw_meta.items():
            if k not in all_meta:
                add_metadata_value(all_meta, k, v)

    # Step 5: Basic file system metadata (always appended last)
    all_meta['SourceFile'] = image_path
    try:
        all_meta['File_Name']      = os.path.basename(image_path)
        all_meta['File_Size']      = os.path.getsize(image_path)
        all_meta['File_Extension'] = ext
    except Exception:
        pass

    print(f'[INFO]  Total merged tags: {len(all_meta)}')
    return all_meta


# ===========================================================================
# Function: load_field_mapping()
# ===========================================================================
def load_field_mapping(search_folder: Path):
    """
    Load Metadata-Field-Mapping.csv from search_folder if it exists.

    The mapping CSV must be in the SAME FOLDER as this .py script so it
    is always found regardless of which image folder is being processed.

    CSV columns:
        Metadata Field          : Raw tag key used internally (e.g. 'Make').
        Windows Properties Name : Human-friendly display name (e.g. 'Camera Make').
        Category                : Grouping label (e.g. 'Camera', 'Origin', 'GPS').

    Data structures returned:
        field_order     (list):  [(metadata_field, windows_name, category), ...]
                                 Ordered list of all fields as they appear in the CSV.
        fields_by_category (dict): {category: [(metadata_field, windows_name), ...]}
                                 Fields grouped by their category label.
                                 Preserves the order within each category.
        category_order  (list):  Category names in first-appearance order.

    category_order is maintained separately (not derived from fields_by_category
    keys) because Python dicts are ordered since 3.7 but this makes the intent
    explicit.

    Returns None (not an empty dict) when the file is absent or unreadable,
    so callers can check 'if not field_mapping' to skip organised outputs.

    Args:
        search_folder (Path): Folder to search for Metadata-Field-Mapping.csv.

    Returns:
        dict | None: Mapping data structure, or None if unavailable.
    """
    mapping_path = search_folder / 'Metadata-Field-Mapping.csv'

    if not mapping_path.is_file():
        print('[WARN]  Metadata-Field-Mapping.csv not found.')
        print('[WARN]  organized_summary.csv and metadata_index.html will be skipped.')
        return None

    field_order        = []     # [(metadata_field, windows_name, category), ...]
    fields_by_category = {}     # {category: [(metadata_field, windows_name), ...]}
    category_order     = []     # Category names in first-seen order

    try:
        with open(mapping_path, 'r', encoding='utf-8') as f:
            reader = csv.DictReader(f)

            for row in reader:
                metadata_field = (row.get('Metadata Field')          or '').strip()
                windows_name   = (row.get('Windows Properties Name') or '').strip()
                category       = (row.get('Category')                or '').strip() \
                                 or 'Uncategorized'

                if not metadata_field:
                    continue    # Skip rows where the key field is blank

                field_order.append((metadata_field, windows_name, category))

                # Build category group, preserving first-appearance order
                if category not in fields_by_category:
                    fields_by_category[category] = []
                    category_order.append(category)

                fields_by_category[category].append((metadata_field, windows_name))

        print(f'[INFO]  Loaded field mapping: {mapping_path}')
        print(f'[INFO]  Fields: {len(field_order)}  |  '
              f'Categories: {len(fields_by_category)}')

        return {
            'field_order'       : field_order,
            'fields_by_category': fields_by_category,
            'category_order'    : category_order,
        }

    except Exception as e:
        print(f'[ERROR] Failed to read Metadata-Field-Mapping.csv: {e}')
        traceback.print_exc()
        return None


# ===========================================================================
# Output helper functions
# ===========================================================================

def format_value(v) -> str:
    """
    Format a metadata value to a plain string suitable for CSV and HTML.

    list / tuple / set values (e.g. GPS coordinates as a tuple of tuples)
    are joined with ' | ' as a separator so they render as a single readable
    cell value.

    None becomes an empty string '' (not the literal string 'None').

    Args:
        v: Any Python value from a metadata dictionary.

    Returns:
        str: Formatted string representation.
    """
    if isinstance(v, (list, tuple, set)):
        return ' | '.join(str(x) for x in v)
    return '' if v is None else str(v)


def _normalize_key_for_match(s: str) -> str:
    """
    Normalise a metadata key for fuzzy matching against mapping entries.

    Removes all characters that vary between libraries for the same logical tag:
        spaces      -> ''   ('EXIF ExposureTime' becomes 'EXIFExposureTime')
        dots        -> ''   ('Image.Make' becomes 'ImageMake')
        colons      -> ''   ('GPS:Latitude' becomes 'GPSLatitude')
        underscores -> ''   ('exposure_time' becomes 'exposuretime')

    Final .lower() makes matching case-insensitive.

    Result: 'ExposureTime', 'EXIF ExposureTime', 'exposure_time' all
    normalise to 'exposuretime' and match the same mapping row.

    Args:
        s (str): Any metadata key string.

    Returns:
        str: Normalised lowercase string with punctuation removed.
    """
    return (
        str(s)
        .lower()
        .replace(' ',  '')
        .replace('.',  '')
        .replace(':',  '')
        .replace('_',  '')
    )


def get_mapped_value(meta: dict, metadata_field: str, windows_name: str):
    """
    Resolve the value for a mapping row from the metadata dict.

    Four-step lookup in priority order:
        1. Exact match on metadata_field   (raw tag key from mapping CSV).
        2. Exact match on windows_name     (display name from mapping CSV).
        3. Fuzzy match on metadata_field   (after _normalize_key_for_match).
        4. Fuzzy match on windows_name     (after _normalize_key_for_match).

    Fuzzy matching builds a normalised lookup dict from all keys in meta
    once per call (norm_map), then checks whether the normalised field or
    name appears.  This handles all the inter-library key name variations
    without needing individual aliases in the mapping CSV.

    Returns '' (empty string) if the key cannot be found by any method,
    so callers always receive a string and never None.

    Args:
        meta            (dict): Merged metadata dict for one image.
        metadata_field  (str):  'Metadata Field' value from the mapping CSV.
        windows_name    (str):  'Windows Properties Name' from the mapping CSV.

    Returns:
        Metadata value if found; '' otherwise.
    """
    # Step 1: exact match on the raw metadata key
    if metadata_field and metadata_field in meta:
        return meta[metadata_field]

    # Step 2: exact match on the Windows display name
    if windows_name and windows_name in meta:
        return meta[windows_name]

    # Build a normalised version of every key in meta for fuzzy comparison
    # Done once per call; not cached because meta varies per image
    norm_map = {}
    for k, v in meta.items():
        nk = _normalize_key_for_match(k)
        if nk not in norm_map:          # first-key-wins for normalised duplicates
            norm_map[nk] = v

    # Step 3: fuzzy match on metadata_field
    if metadata_field:
        nmf = _normalize_key_for_match(metadata_field)
        if nmf in norm_map:
            return norm_map[nmf]

    # Step 4: fuzzy match on windows_name
    if windows_name:
        nwn = _normalize_key_for_match(windows_name)
        if nwn in norm_map:
            return norm_map[nwn]

    return ''   # Not found by any method


# ===========================================================================
# Function: create_comprehensive_summary_csv()
# ===========================================================================
def create_comprehensive_summary_csv(dest_folder: Path,
                                      all_metadata_list: list) -> None:
    """
    Write comprehensive_summary.csv: one column per discovered EXIF key.

    Header construction:
        - Collects ALL keys across ALL images into a set.
        - Sorts alphabetically for consistent column order across runs.
        - Moves 'SourceFile' to be the FIRST column (if present) so the
          image filename is always in column A.

    set.update(meta.keys())
        Adds all keys from one image's dict to the master key set.
        The set automatically deduplicates keys that appear in multiple images.

    Also writes metadata_fields_reference.txt beside the CSV: a plain list
    of every field name for reference when building Metadata-Field-Mapping.csv.

    Args:
        dest_folder        (Path):       Timestamped output folder.
        all_metadata_list  (list[dict]): One dict per image.
    """
    if not all_metadata_list:
        print('[WARN]  No metadata to write for comprehensive_summary.csv.')
        return

    summary_path = dest_folder / 'comprehensive_summary.csv'

    # Collect every key from every image into one sorted list
    all_keys = set()
    for meta in all_metadata_list:
        all_keys.update(meta.keys())
    all_keys = sorted(all_keys)     # Alphabetical order for consistency

    # Promote SourceFile to first column for readability
    if 'SourceFile' in all_keys:
        all_keys.remove('SourceFile')
        header = ['SourceFile'] + all_keys
    else:
        header = all_keys

    try:
        with open(summary_path, 'w', encoding='utf-8', newline='') as f:
            writer = csv.writer(f)
            writer.writerow(header)

            for meta in all_metadata_list:
                # meta.get(col, '') returns '' for images missing a tag
                row = [format_value(meta.get(col, '')) for col in header]
                writer.writerow(row)

        print(f'[INFO]  comprehensive_summary.csv: {summary_path}')
        print(f'[INFO]  Rows: {len(all_metadata_list)}  |  '
              f'Columns: {len(header)}')

        # Write the field reference list beside the CSV
        ref_path = dest_folder / 'metadata_fields_reference.txt'
        with open(ref_path, 'w', encoding='utf-8') as f:
            f.write('COMPREHENSIVE METADATA FIELDS REFERENCE\n')
            f.write('=' * 50 + '\n\n')
            for col in header:
                f.write(col + '\n')
        print(f'[INFO]  metadata_fields_reference.txt: {ref_path}')

    except Exception as e:
        print(f'[ERROR] Error writing comprehensive_summary.csv: {e}')
        traceback.print_exc()


# ===========================================================================
# Function: create_organized_summary_by_category()
# ===========================================================================
def create_organized_summary_by_category(dest_folder: Path,
                                          all_metadata_list: list,
                                          field_mapping) -> None:
    """
    Write organized_summary.csv with columns ordered by mapping category.

    Columns are derived from Metadata-Field-Mapping.csv category order:
        - SourceFile is always the first column.
        - Remaining columns appear in the order the categories and fields
          appear in the mapping file (not alphabetical).

    Display name selection:
        windows_name or metadata_field
        If the mapping row has a Windows Properties Name, that is used as
        the column header; otherwise the raw Metadata Field name is used.

    get_mapped_value() handles the fuzzy lookup for each field per image.

    Skipped silently if field_mapping is None (mapping file was not found).

    Args:
        dest_folder        (Path):       Timestamped output folder.
        all_metadata_list  (list[dict]): One dict per image.
        field_mapping      (dict|None):  From load_field_mapping(); None to skip.
    """
    if not all_metadata_list:
        print('[WARN]  No metadata to write for organized_summary.csv.')
        return

    if not field_mapping:
        print('[INFO]  No mapping loaded -- skipping organized_summary.csv.')
        return

    organized_path = dest_folder / 'organized_summary.csv'

    # Build the ordered column list from the mapping
    header   = ['SourceFile']
    col_defs = []   # (metadata_field, windows_name, display_name)

    for category in field_mapping['category_order']:
        fields = field_mapping['fields_by_category'].get(category, [])
        for metadata_field, windows_name in fields:
            display = windows_name or metadata_field    # Prefer friendly name
            col_defs.append((metadata_field, windows_name, display))
            header.append(display)

    try:
        with open(organized_path, 'w', encoding='utf-8', newline='') as f:
            writer = csv.writer(f)
            writer.writerow(header)

            for meta in all_metadata_list:
                row = [format_value(meta.get('SourceFile', ''))]
                for metadata_field, windows_name, _display in col_defs:
                    value = get_mapped_value(meta, metadata_field, windows_name)
                    row.append(format_value(value))
                writer.writerow(row)

        print(f'[INFO]  organized_summary.csv: {organized_path}')
        print(f'[INFO]  Rows: {len(all_metadata_list)}  |  '
              f'Columns: {len(header)}')

    except Exception as e:
        print(f'[ERROR] Error writing organized_summary.csv: {e}')
        traceback.print_exc()


# ===========================================================================
# Function: create_html_metadata_index()
# ===========================================================================
def create_html_metadata_index(dest_folder: Path,
                                all_metadata_list: list,
                                field_mapping) -> None:
    """
    Write metadata_index.html: a styled HTML table with one row per image.

    Columns in the HTML table:
        #       : Row index number.
        File    : Filename as a clickable link to the ORIGINAL image.
        Camera  : All fields in the 'Camera' category from the mapping.
        Date    : The first 'Origin' category field whose name contains
                  'datetaken' or 'datetimeoriginal' (case-insensitive).

    file:// URL construction:
        Path(src).as_uri()
            Converts an absolute filesystem path to a standards-compliant
            file:// URI.  On Windows 'C:\\Photos\\img.jpg' becomes
            'file:///C:/Photos/img.jpg'.  On Unix '/home/user/img.jpg'
            becomes 'file:///home/user/img.jpg'.

    Sticky table header:
        CSS 'position: sticky; top: 0' keeps the header row visible when
        the user scrolls down through a long table.

    Skipped if field_mapping is None or all_metadata_list is empty.

    Args:
        dest_folder        (Path):       Timestamped output folder.
        all_metadata_list  (list[dict]): One dict per image.
        field_mapping      (dict|None):  From load_field_mapping(); None to skip.
    """
    if not all_metadata_list:
        print('[WARN]  No metadata to write for metadata_index.html.')
        return

    if not field_mapping:
        print('[INFO]  No mapping loaded -- skipping metadata_index.html.')
        return

    html_path = dest_folder / 'metadata_index.html'

    # Determine which camera and date fields to show
    camera_fields = field_mapping['fields_by_category'].get('Camera', [])
    key_fields    = []

    # Find the first Origin field that is a 'date taken' / 'datetime original'
    origin_fields = field_mapping['fields_by_category'].get('Origin', [])
    for field, name in origin_fields:
        lname = (name or '').lower()
        if 'datetaken' in lname or 'datetimeoriginal' in field.lower():
            key_fields.append((field, name))
            break   # Only use the first matching date field

    try:
        with open(html_path, 'w', encoding='utf-8') as f:

            # HTML document structure with embedded CSS
            # {{ }} inside f-string produces literal { } for CSS rules
            f.write("""<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Image Metadata Index</title>
    <style>
        body {
            font-family: Segoe UI, Tahoma, sans-serif;
            background-color: #f5f5f5;
            padding: 20px;
        }
        h1 {
            text-align: center;
            color: #333;
            border-bottom: 3px solid #4CAF50;
            padding-bottom: 10px;
        }
        table {
            width: 100%;
            border-collapse: collapse;
            background: white;
            box-shadow: 0 2px 6px rgba(0,0,0,0.1);
        }
        th, td {
            padding: 6px 8px;
            border: 1px solid #ddd;
            font-size: 12px;
        }
        th {
            background-color: #4CAF50;
            color: white;
            position: sticky;
            top: 0;
            z-index: 2;
        }
        tr:nth-child(even) { background-color: #fafafa; }
        tr:hover           { background-color: #f1f1f1; }
        .index-cell    { text-align: right; color: #555; width: 40px; }
        .filename-link { color: #0078D4; text-decoration: none; }
        .filename-link:hover { text-decoration: underline; }
    </style>
</head>
<body>
    <h1>Image Metadata Index</h1>
    <p>Generated by Comprehensive EXIF Data Extractor - Chatbizdb.com</p>
    <div style="overflow-x:auto;">
    <table>
        <thead>
            <tr>
                <th>#</th>
                <th>File</th>
""")

            # Camera column headers
            for _field, win_name in camera_fields:
                header = win_name or _field
                f.write(f'                <th>{header}</th>\n')

            # Date column headers
            for _field, win_name in key_fields:
                header = win_name or _field
                f.write(f'                <th>{header}</th>\n')

            f.write('            </tr>\n        </thead>\n        <tbody>\n')

            # Data rows -- one per image
            for idx, meta in enumerate(all_metadata_list, start=1):
                src      = meta.get('SourceFile', '')
                filename = os.path.basename(src)

                # Build a file:// URL to the ORIGINAL image
                try:
                    file_url = Path(src).as_uri() if src else ''
                except Exception:
                    file_url = ''

                f.write('            <tr>\n')
                f.write(f"                <td class='index-cell'>{idx}</td>\n")
                f.write(
                    f"                <td>"
                    f"<a href='{file_url}' class='filename-link' "
                    f"target='_blank'>{filename}</a></td>\n"
                )

                # Camera metadata columns
                for field, win_name in camera_fields:
                    v = format_value(get_mapped_value(meta, field, win_name))
                    f.write(f'                <td>{v}</td>\n')

                # Date metadata columns
                for field, win_name in key_fields:
                    v = format_value(get_mapped_value(meta, field, win_name))
                    f.write(f'                <td>{v}</td>\n')

                f.write('            </tr>\n')

            f.write("""        </tbody>
    </table>
    </div>
    <div style="text-align: center; margin-top: 16px;
                color: #666; font-size: 0.85em;">
        <p>Column names based on Metadata-Field-Mapping.csv.</p>
    </div>
</body>
</html>
""")

        print(f'[INFO]  metadata_index.html: {html_path}')

    except Exception as e:
        print(f'[ERROR] Error writing metadata_index.html: {e}')
        traceback.print_exc()


# ===========================================================================
# Function: open_html_report()
# ===========================================================================
def open_html_report(dest_folder: Path) -> None:
    """
    Open metadata_index.html in the OS default browser or viewer.

    Platform-specific approach:
        Windows : os.startfile(path)
                  Uses the Windows ShellExecute API; opens whatever
                  app is registered for .html files (usually a browser).
        macOS / Linux : webbrowser.open(uri)
                  Uses Python's built-in webbrowser module which delegates
                  to the OS default browser via xdg-open on Linux or the
                  macOS 'open' command.

    Path.resolve().as_uri()
        resolve() converts to an absolute path (resolving any symlinks).
        as_uri() produces a standards-compliant file:// URL for the browser.

    sys.platform.startswith('win')
        Returns True on Windows ('win32' or 'win64').
        Using startswith is more robust than sys.platform == 'win32'.

    Args:
        dest_folder (Path): Timestamped output folder containing the HTML.
    """
    html_path = dest_folder / 'metadata_index.html'

    if not html_path.is_file():
        print('[INFO]  metadata_index.html not found -- nothing to open.')
        return

    try:
        if sys.platform.startswith('win'):
            os.startfile(str(html_path))    # Windows ShellExecute
        else:
            import webbrowser
            webbrowser.open(html_path.resolve().as_uri())

        print(f'[INFO]  Opened HTML report: {html_path}')

    except Exception as e:
        print(f'[WARN]  Could not open HTML report automatically: {e}')
        print(f'[INFO]  Manual path: {html_path}')


# ===========================================================================
# Function: main()
# ===========================================================================
def main() -> None:
    """
    Orchestrate the full EXIF extraction and report generation workflow.

    Steps:
        1. User selects the image folder via GUI dialog or console input.
        2. Derive the TOP folder (parent of selected) and create Report/ root.
        3. Create a timestamped EXIF-Summary-* output folder inside Report/.
        4. Load Metadata-Field-Mapping.csv from the program folder (optional).
        5. Discover all image/RAW files inside the selected image folder.
        6. Extract metadata from every image using all available libraries.
        7. Write comprehensive_summary.csv + metadata_fields_reference.txt.
        8. Write organized_summary.csv (requires mapping).
        9. Write metadata_index.html (requires mapping).
        10. Open metadata_index.html in the default browser.
    """
    # Step 1 -- Folder selection
    images_folder = select_folder()

    # Step 2 -- Derive TOP and Report folders
    # images_folder.parent is the TOP folder (one level above the image folder)
    top_folder  = images_folder.parent
    report_root = top_folder / 'C-Reports'
    report_root.mkdir(parents=True, exist_ok=True)

    print(f'[INFO]  TOP folder    : {top_folder}')
    print(f'[INFO]  Report folder : {report_root}')

    # Step 3 -- Create the timestamped output subfolder
    dest_folder = create_timestamped_folder(report_root, prefix='exif-report')

    # Step 4 -- Load the field mapping from the PROGRAM folder (script location)
    # Using the script's folder (not the image folder) makes the mapping file
    # portable: the same mapping works for any image folder selected by the user.
    script_folder = Path(__file__).resolve().parent
    field_mapping = load_field_mapping(script_folder)

    # Step 5 -- Discover image files
    image_files = get_image_files(images_folder)
    if not image_files:
        print('[ERROR] No supported image files found. Exiting.')
        return

    # Step 6 -- Extract metadata from every image
    all_metadata_list = []
    success           = 0

    for img in image_files:
        try:
            meta = extract_all_metadata(img)
            all_metadata_list.append(meta)
            success += 1
        except Exception:
            print(f'[ERROR] Failed to process: {img}')
            traceback.print_exc()

    if not all_metadata_list:
        print('[ERROR] No metadata extracted from any file. Exiting.')
        return

    print(f'\n[INFO]  Extracted metadata: {success} / {len(image_files)} files.')

    # Step 7 -- Comprehensive CSV + field reference
    create_comprehensive_summary_csv(dest_folder, all_metadata_list)

    # Step 8 -- Organised CSV (only if mapping was loaded)
    create_organized_summary_by_category(dest_folder, all_metadata_list,
                                         field_mapping)

    # Step 9 -- HTML index (only if mapping was loaded)
    create_html_metadata_index(dest_folder, all_metadata_list, field_mapping)

    # Step 10 -- Open the HTML report
    open_html_report(dest_folder)

    print('\n[DONE]  All outputs written.')
    print(f'[INFO]  Output folder: {dest_folder}')


# ===========================================================================
# Entry-point guard
# ===========================================================================
if __name__ == '__main__':
    main()
